Tracking a motion of a jaw

ABSTRACT

The invention relates to tracking motion of a patient&#39;s jaw, wherein motion of a tracking item which represents motion of the jaw is followed by at least one camera arranged to a medical x-ray imaging apparatus and wherein the motion detected by the at least one camera is applied on a digital model depicting hard tissue of the jaw. The moving digital model of the hard tissue of the jaw thus generated is shown on a display to visualize movement of the hard tissue.

The application claims the benefit of U.S. Provisional Application Ser.No. 62/130,447 filed Mar. 9, 2015, the disclosure of which is hereinincorporated by reference.

FIELD OF INVENTION

The invention relates to arrangements and methods for tracking andgenerating a digital model visualizing motion of hard tissue of aperson's jaw.

BACKGROUND OF INVENTION

Various systems such as ones employing mechanical, electronic,ultrasonic, electromagnetic and optical techniques have been used inconnection with recording movements of a human jaw. One typical solutionincludes attaching physical markers to both maxilla and mandible andrecoding their respective relative motion. Such systems are known toinclude visualization of the measured or detected movement on a display,possibly as applied to e.g. a digital surface model of cranial hardtissue.

Regardless the technology, the work-flow involved in these procedures isoften time-consuming and laborious as it may include using separateapparatus and operations performed individually and separate from eachother. These operations may include attaching markers to the anatomy,generating jaw movements and detecting and recording the movements ofthe markers, and generating a model visualizing the jaw movements whichmay then be shown on a display. For one, in case modelling of the jawmovements is visualized as a separate process afterwards and it is onlythen realized that additional movement information would be needed or bedesirable, to enhance the digital motion model, this will not bepossible until the next time one will be able to harness the patientwith the markers and detect and record the jaw movements anew.

Examples of the prior art to track jaw motions include systems describedin patent publications U.S. Pat. No. 4,836,778, U.S. Pat. No. 4,859,181,US 2013/0157218 and WO 2013/0175018.

BRIEF DESCRIPTION OF INVENTION

The invention and its preferable embodiments include an arrangement andmethod in which a digital model showing a person's jaw motions isgenerated in the context of a medical x-ray imaging apparatus, such asCT apparatus arranged to acquire image information for generatingsurface models of cranial hard tissue anatomies. This enables acquiringboth x-ray image data and jaw motion data with the same apparatus. Byequipping the arrangement with image processing means and a display onwhich a modelled motion of the hard tissue can be visualized in the samecontext in which photographing the motion of reference objects attachedto a cranial anatomy is carried out, embodiments of the invention enablefollowing the movements of the hard tissue in real time while thepatient is present to generate any desired chewing or other jaw motionto be modelled.

According to one embodiment, the method according to the inventionincludes providing at least one camera arranged in physical connectionwith a CT imaging apparatus. The method includes attaching a firsttracking item to a person's jaw and attaching a second tracking item toa person's upper jaw or to a part of the person's anatomy which is in astationary connection with the upper jaw. A CT reconstruction of hardtissue of anatomy of the person is generated and locations of thetracking items relative to the hard tissue anatomy are identified. Inone embodiment, a series of images of the tracking items using the atleast one camera are captured while the patient is performing a jawmotion and locations of the tracking items in the images are determined.The poses of the hard tissue anatomy are then resolved using knowledgeof a relationship between the tracking items and the anatomy. The posesof the hard tissue anatomies can be transferred to a visual system todisplay the digital model chewing motion of the jaw.

BRIEF DESCRIPTION OF FIGURES

The following Figs. are used to present some aspects of variousembodiments of the invention.

FIG. 1 shows basic parts of one preferable arrangement according to theinvention.

FIG. 2 shows a principled arrangement for detecting jaw motion accordingone system applicable for use in the context of the invention.

FIG. 3 shows a detector module attachable to a medical x-ray apparatusincluding components applicable for use in the context of the invention.

FIG. 4 shows an arrangement for motion tracking imaging in which atracking device is attached to a mandible, in the field of view of twocameras.

FIG. 5 shows an arrangement for motion tracking imaging as in FIG. 4 butin which only a portion of reference objects of the tracking device arein the field of view of either of two cameras.

FIGS. 6 and 7 show procedures (by flow chart) applicable for use in theinvention for generating a digital model visualizing jaw motion.

DETAILED DESCRIPTION OF INVENTION

FIG. 1 shows basic parts of one preferable arrangement according to theinvention. The arrangement of FIG. 1 includes a medical x-ray imagingapparatus (10) comprising a vertical support construction (11) fromwhich horizontally extends an arm (12) supporting a patient supportmeans (17) and an arm part (13) which supports an arm part (14)supporting imaging means of the apparatus. The arm part (14) supportingthe imaging means is arranged rotatable. The imaging means of theapparatus include an x-ray source (15) and a receiver of x-ray imageinformation (21) arranged at a distance from each other. The imagingmeans are located with respect to the patient support means (17) suchthat an imaging station (18) is created within an area between the x-raysource (15) and the receiver of x-ray image information (21) such that abeam generated by the x-ray source (15) can be directed through saidimaging station (18) towards the receiver of x-ray image information(21). The apparatus includes control means of which FIG. 1 shows a userinterface (16) arranged to the arm (12) supporting a patient supportmeans (17) and an operation mode selection means (19) pertaining in it.In the apparatus according to FIG. 1, the receiver of x-ray imageinformation (21) is arranged as part of a receiver module of imageinformation (20), a detector module which is arranged in an operationalconnection with a computer (30). A means for processing imageinformation is arranged to the computer (30) and the computer which isalso arranged in an operational connection with a display (31). The userinterface (16) may be equipped with a display, too, and there may be adisplay arranged to some other structure of the x-ray imaging apparatusas well. Physical components and subsystems of the control system (CS)may be arranged at various places of the arrangement, some may beincluded in the computer (30) and some be arranged e.g. in the verticalsupport construction (11) or in the proximity or integrated with thecomponent of the arrangement to be controlled.

FIG. 2 shows a principled arrangement of detecting jaw motion accordingto one system applicable for use in the context of the invention. Thearrangement includes two cameras (22) arranged at a distance from eachother and aimed to photograph a person's head whereto tracking means(50) are connected. In the arrangement according to FIG. 2, the trackingmeans (50) consist of two separate support constructions (51) forreference objects such as light reflecting objects (52). One of thesupport constructions (51) is connected to a person's forehead, anotherto a person's lower jaw. The arrangement of FIG. 2 also includes lightsources (23) arranged at close proximity of the cameras (22). The lightsource (23) is arranged to emit light essentially in a direction atwhich the proximate camera (22) is aimed.

FIG. 3 shows a detector module (20) attachable to a medical x-rayapparatus including components applicable for use in the context of theinvention. Contrary to what has been presented in FIG. 2, the cameras(22) are arranged to the module (20) not at a vertical but at ahorizontal distance from each other, and there are light sources (23)above and below the cameras (22). An x-ray image detector (21) is alsoarranged in the module. Attaching of the module (20) to the medicalx-ray apparatus is to be realized such that the x-ray image detector(21) is or can be aligned at the imaging station (18) of the apparatus.

FIG. 4 shows an arrangement for motion tracking imaging in which atracking means is attached to a mandible and in the field of view of twocameras. The tracking means (50) of FIG. 4 comprises a support structure(51) for five light-reflecting reference objects (52). The two cameras(22) of the arrangement are located and aimed such with respect to thelight-reflecting reference objects (52) that all five of them are in theline of sight of both of the cameras (22). The arrangement includes alight source (23) at close proximity of both of the cameras (22), thelight sources (23) being arranged to emit light essentially in adirection at which the proximate camera (22) is aimed.

FIG. 5 shows an arrangement for motion tracking imaging as in FIG. 4 butin which the cameras (22) are located at a longer distance from eachother. Such arrangement may be applied e.g. in the context like the oneshown in FIG. 3, in which the cameras (22) are located on opposite sidesof the x-ray image detector (21) of the detector module (20).

In the arrangement of FIG. 5, only a portion of the reference objects(52) of the tracking means (50) are in the field of view of either ofthe two cameras (22), but resolving the pose of the tracking means (50)is possible also in the context of such arrangement, as long as at leastone of the reference objects (52) is in the field of view of bothcameras (22). Such procedure may comprise resolving, first, the threedimensional position of one reference object (52) in the field of viewof both cameras. This can be done by e.g. triangulating the positionusing the camera calibration information and detected locations of thereference objects (52), after which a corresponding point in a trackingdevice model is translated into the resolved position. The trackingdevice model is then rotated about this point so that when projectedinto both camera views, the squared distance between the projected pointposition and the corresponding detected reference object position isminimized. The rotation and the translation define the pose of thetracking model.

The arrangements for acquiring information for generating a digitalmodel showing motion of a cranial hard tissue anatomy discussed aboveare based on using two cameras, but the invention can be implementedusing arrangements based on using any number of cameras.

FIG. 6 shows one procedure applicable for use in the arrangement of theinvention for visualizing jaw motion. When a person positioned forimaging by at least one camera arranged to a medical x-ray imagingapparatus starts jaw motion, tracking of the motion is initiated bytaking photographs of at least one tracking item (in reference to theabove, jaw tracking means comprise light reflecting referenceobject(s)). The control system of the arrangement then detects in aphotograph said at least one tracking item and resolves its pose basedon a reference model of the tracking item and camera calibrationinformation. A modelled jaw movement can then be visualized on a displayeven in real time by repeatedly transforming the resolved pose of thetracking item to a digital model of the anatomy.

One method for producing the reference model of the tracking meansincludes first using the medical x-ray imaging apparatus for generatinga CT reconstruction of a mandible and the tracking means includingreference objects, and a visualization of that reconstruction is thenpresented on a display. When the user interface of the arrangementincludes means to point locations of the reference objects on thedisplayed visualization, the image processing software of thearrangement is able to generate the reference model defining the spatialrelationship between the reference objects and the reconstruction of theanatomy.

The camera calibration information referred to in FIG. 6 includes theintrinsic and extrinsic parameters of the one or more cameras of thearrangement. Such calibration information can be acquired using standardcomputer vision camera calibration methods known to those skilled in theart.

FIG. 7 shows another way of presenting an embodiment of the invention.According FIG. 7, a tracking item comprising reference objects isattached to a person's lower jaw and a second tracking item is attachedto a part of the person's anatomy which is in a stationary connectionwith the person's upper jaw. Then, a CT reconstruction is generatedcomprising hard tissue of the anatomies to which the tracking items areattached and locations of the reference objects in relation to the hardtissue anatomies are determined. When a person performs a jaw motion,one or more than one camera arranged to a CT imaging apparatus is usedto capture a series of images of the reference objects during the jawmotion and locations of the reference objects are detected from thecaptured images. Poses of the hard tissue anatomies are then resolvedusing the detected locations of the reference objects and the determinedrelation between locations of the reference objects and the hard tissueanatomies and the resolved poses are transferred to a visualizationsystem to show a digital model of motion of hard tissue anatomiesaccording to the person's jaw motion.

In summary, various aspects of the invention can be consideredcomprising tracking motion of a person's jaw by at least one cameraconfigured to photograph movement of tracking items, the tracking itemscomprising reference objects and being attached to a person's lover jawand to a person's upper jaw or to a part of the person's anatomy whichis in a stationary connection with the upper jaw. An arrangement can beused comprising a control system including a first subsystem forcontrolling operations of the at least one camera to photograph themovement of the tracking items, and a second subsystem including imageinformation processing means to detect projections and resolve poses ofthe tracking item in the images captured by the at least one camera andto apply this information to a digital model depicting hard tissue of atleast the lower jaw and to generate a moving digital model to be shownon a display which visualizes movement of the hard tissue of the lowerjaw according to the photographed movement of the reference objects. Thearrangement can further comprise a medical x-ray imaging apparatusincluding an x-ray source and an x-ray image detector and the controlsystem of the arrangement further comprise a third subsystem, the thirdsubsystem including control functions relating to controlling operationsof the medical x-ray imaging apparatus to acquire cranial x-ray imageinformation and for generating a CT reconstruction of a cranial anatomy.The at least one camera to photograph the movement of the tracking itemscan be arranged to be a physical part of the medical x-ray imagingapparatus.

As to the medical x-ray apparatus, it can include a first structurecarrying the x-ray source and the x-ray image detector and a secondstructure supporting the first structure. The at least one camera can beconnected to either of the first structure and the second structure.Preferably, the arrangement includes two cameras arranged at a distancefrom each other on the first structure carrying said x-ray source andx-ray image detector. Further, the medical x-ray imaging apparatus caninclude a light source arranged at a close proximity of said at leastone camera and the light source to be arranged to emit light essentiallyin a direction at which said at least one camera is aimed.

The display of the arrangement can be connected to structures of themedical x-ray imaging apparatus and the control system be configured tovisualize on the display the digital model showing movement of the hardtissue according to said photographed movement detected by the at leastone camera. Preferably, the control system is configured to visualizethe movement of the digital model generated by the second subsystemessentially simultaneously while said first subsystem operates said atleast one camera to photograph the movement of the reference objects.The digital model depicting hard tissue can be a visualization of a CTreconstruction of a person's jaw or jaws generated based on image datadetected by the x-ray image detector of the medical x-ray imagingapparatus.

According to one embodiment, a user interface arranged in connectionwith the control system is configured to enable marking a feature ofinterest on the digital model depicting hard tissue and to include thismarking in said moving digital model. The control system can further beconfigured to present on the display an image which is a visualizationof a CT reconstruction of at least a part of the person's cranialanatomy together with a tracking item including reference objectsattached to the person when image information for said reconstructionwas acquired and, as a response to a user input indicating locations ofthe reference objects of said tracking item on said image, to generate areference model defining a spatial relationship between said referenceobjects and the CT reconstruction.

The present disclosure advantageously combines an x-ray imagingapparatus with a camera system for tracking jaw motion physicallyarranged thereto. This improves the work flow of the imaging procedureand allows control of the CT apparatus and the motion tracking camerasystem by the same control system.

By arranging the jaw motion camera system to the x-ray imaging apparatusitself the present process provides the possibility to both generate aCT reconstruction which includes the anatomy and the tracking items(that is, scanning the anatomy while the tracking items are attached toa person) and to track jaw motion within the same procedure while aperson is present and available for imaging at the x-ray imagingapparatus. Therefore, for one, no recalling of the person for imagingand no re-attaching of the tracking items is needed in case one noticesany flaw in the process or problems in the image quality as the personis still present for any retake or performing additional jaw motions.

It will be apparent to one skilled in the art that as for its details,the present invention may be implemented also in other ways thanaccording to the embodiments of the invention described above and thatvarious details of the embodiments may be realized also in othercombinations than ones literally discussed above. As an example, themedical x-ray imaging apparatus needs not to be exactly like the oneshown in FIG. 1. The x-ray imaging means of the medical x-ray imagingapparatus may be arranged to, instead of a support arm, e.g. within aring-shaped gantry.

1. Arrangement for tracking motion of a person's jaw, the arrangementcomprising: at least one camera configured to photograph movement oftracking items, the tracking items comprising reference objects andbeing attached to a person's lover jaw and to a person's upper jaw or toa part of the person's anatomy which is in a stationary connection withthe upper jaw; a control system including a first subsystem forcontrolling operations of said at least one camera to photograph saidmovement of the tracking items, and a second subsystem including imageinformation processing means to detect projections and resolve poses ofthe tracking item in the images captured by said at least one camera andto apply this information to a digital model depicting hard tissue of atleast the lower jaw and to generate a moving digital model to be shownon a display which visualizes movement of the hard tissue of the lowerjaw according to said photographed movement of the reference objects;wherein the arrangement further comprises a medical x-ray imagingapparatus including an x-ray source and an x-ray image detector and thecontrol system further comprises a third subsystem, the third subsystemincluding control functions relating to controlling operations of saidmedical x-ray imaging apparatus to acquire cranial x-ray imageinformation and for generating a CT reconstruction of a cranial anatomy,characterized in that said at least one camera arranged to photographsaid movement of the tracking items is a physical part of said medicalx-ray imaging apparatus.
 2. Arrangement according to claim 1, whereinsaid medical x-ray apparatus includes a first structure carrying saidx-ray source and x-ray image detector and a second structure supportingthe first structure, and wherein said at least one camera is connectedto either of the first structure and the second structure. 3.Arrangement according to claim 2, wherein the arrangement includes twocameras arranged at a distance from each other to said first structurecarrying said x-ray source and x-ray image detector.
 4. Arrangementaccording to claim 1, wherein said medical x-ray imaging apparatusincludes a light source arranged at a close proximity of said at leastone camera and said light source is arranged to emit light essentiallyin a direction at which said at least one camera is aimed. 5.Arrangement according to claim 1, wherein said display is connected tostructures of the medical x-ray imaging apparatus and said controlsystem is configured to visualize on the display said digital modelshowing movement of the hard tissue according to said photographedmovement detected by said at least one camera.
 6. Arrangement accordingto claim 1, wherein the control system is configured to visualize saidmovement of the digital model generated by said second subsystemessentially simultaneously while said first subsystem operates said atleast one camera to photograph the movement of the reference objects. 7.Arrangement according to claim 1, wherein said digital model depictinghard tissue is a visualization of a CT reconstruction of a person's jawor jaws generated based on image data detected by the x-ray imagedetector of said medical x-ray imaging apparatus.
 8. Arrangementaccording to claim 1 further comprising a user interface in connectionwith said control system, wherein the user interface is configured toenable marking a feature of interest on said digital model depictinghard tissue and wherein the control system is configured to include thismarking in said moving digital model.
 9. Arrangement according to claims1, wherein the control system is configured to present on said displayan image which is a visualization of a CT reconstruction of at least apart of the person's cranial anatomy together with a tracking itemincluding reference objects attached to the person when imageinformation for said reconstruction was acquired and, as a response to auser input indicating locations of the reference objects of saidtracking item on said image, to generate a reference model defining aspatial relationship between said reference objects and the CTreconstruction.
 10. Arrangement according to claim 2, wherein said firststructure is a rotatable arm upon which the x-ray source and the x-rayimage detector are arranged at a distance from each other and whereinsaid at least one camera is arranged on said rotatable arm adjacent saidx-ray image detector.
 11. Arrangement according to claim 1, wherein thereference objects are light reflecting.
 12. Method for tracking motionof a person's jaw, comprising: attaching a first tracking itemcomprising reference objects to a person's lower jaw and a secondtracking item comprising reference objects to a part of the person'sanatomy which is in a stationary connection with the person's upper jaw;generating a CT reconstruction comprising hard tissue of the anatomiesto which the tracking items are attached; determining locations of thereference objects in relation to the hard tissue anatomies; capturing aseries of images of the reference objects while a person performs jawmotion; detecting locations of the reference objects from the capturedimages; resolving poses of the hard tissue anatomies using the detectedlocations of the reference objects and the determined relation betweenlocations of the reference objects and the hard tissue anatomies;transferring the resolved poses to a visualization system to show adigital model of motion of the hard tissue anatomies according to theperson's jaw motion; characterized in that said series of images iscaptured by at least one camera arranged to a CT imaging apparatus. 13.Method according to claim 12, wherein said motion of the digital modelof the hard tissue anatomies is visualized essentially simultaneouslywhile said at least one camera captures said images.
 14. Methodaccording to claim 12, wherein said second tracking item is connected tothe patient's forehead.
 15. Method according to claim 12, wherein saidhard tissue anatomy comprises cranial hard tissue.
 16. Method accordingto claim 12, wherein said at least camera and a light source aredisposed on a rotating arm of a CT imaging apparatus.
 17. Methodaccording to claim 16, further comprising illuminating the referenceobjects of the first and the second tracking item by said light sourceduring capturing of the images.
 18. Method according to claim 12,further comprising marking locations of reference objects on said CTreconstruction using a user interface and depicting said locations onthe images.
 19. Method according to claim 12, wherein at least onetracking item includes at least 5 light reflecting objects spaced apartfrom each other.